Add OSSF Scorecard callee workflow#113
Conversation
dblane-digicatapult
left a comment
There was a problem hiding this comment.
Local review of the OSSF Scorecard callee, cross-checked against the live run in digicatapult/hello-world#25 (both scorecard jobs pass there, but only because hello-world is public). Thanks for the thorough README and the fail-fast validation step. Requesting changes on one blocking item plus a few robustness/consistency points.
Blocking
- The
analysisjob permissions omitcontents: read/actions: read/pull-requests: read. The hello-world run confirms the job executes with onlyMetadata: read+SecurityEvents: write, which breaks checkout on private callers and silently degrades several checks on public ones. See inline.
Public vs private
- The file claims public/private support but has no visibility detection or skip, and
publish_results/badge are public-only. Decide one way and make the code match (inline on line 3).
Should fix
repo_tokendeclared as both an input and a secret; the input is dead and contradicts the README table.required: true+default:onresults_file/results_format.- Inputs interpolated directly into the docker
run:block. checkout@v6.0.2/upload-artifact@v7.0.1pin drift vs the rest of the repo.- No guard against
results_format: json+upload_type: dashboard.
Nits (non-blocking)
examples/generate-security-scorecard.md:9typo: "default mod" -> "mode".- Custom-scan passes both
--checksand--policy; the generated policy already disables the unselected checks (and usesscore: 1, so it enforces nothing meaningful), so the two are redundant. Confirmed non-fatal against scorecard'soptions.go. ENABLE_SARIF: ... || ''still registers the env var, so scorecard treats SARIF as enabled even forjson(itsLookupEnvcheck returns set). Harmless, but not what it looks like.upload_typevocabulary (dashboard/artifact) diverges from the other security workflows (sarif/artefact); an unrecognised value silently uploads nothing.
| @@ -0,0 +1,159 @@ | |||
| name: Scorecard analysis workflow | |||
|
|
|||
| # Only use on public/private repositories where Code Scanning has been enabled. | |||
There was a problem hiding this comment.
This says public and private, but nothing in the workflow detects repo visibility or skips accordingly, and the permissions block below only works for public repos today. Two coherent options:
- If private is genuinely in scope: fix the permissions (see the comment on the
permissions:block) so private callers actually work. - If this is public-only in practice (
publish_results/ badge / REST API are all public-only per the README note): guard it, e.g.if: ${{ !github.event.repository.private }}on the job, or fail fast, and reword this line.
As written it is neither: it advertises private support it cannot deliver, with no guard to skip private callers.
| required: false | ||
| inputs: | ||
| results_file: | ||
| required: true |
There was a problem hiding this comment.
required: true combined with a default: is contradictory for workflow_call inputs: required forces the caller to pass a value, so the default is unreachable, yet the README lists these as Required with a default. Set required: false (the defaults are sensible) or drop the defaults. Same applies to results_format just below.
| required: false | ||
| type: string | ||
| default: "archive" # archive or git only | ||
| repo_token: |
There was a problem hiding this comment.
repo_token is declared here as an input and as a secret (lines 6-8). Every step reads secrets.repo_token; this input is never referenced. The hello-world run shows it riding along empty in the Inputs dump while the real token arrived via secrets:. The README Inputs table documents it as a with: input, so a caller who follows the table will have it silently ignored. Drop this input, keep the secret, and move the README row into a Secrets section.
| analysis: | ||
| name: Scorecard analysis | ||
| runs-on: ubuntu-latest | ||
| permissions: |
There was a problem hiding this comment.
Blocking: this token is missing contents: read (plus actions: read, pull-requests: read).
Because this block is an explicit allow-list, the token that reaches analysis is exactly these two scopes. Confirmed in the hello-world run (digicatapult/hello-world#25), where the job's own token dump reads only:
Metadata: read
SecurityEvents: write
Consequences:
Checkout codeclones with this token, so on a private caller repo it fails outright. Public repos clone unauthenticated, which is the only reason hello-world stayed green.- Even on public repos, several requested checks (Branch-Protection, Code-Review, Token-Permissions, Signed-Releases) need
contents/pull-requestsread to evaluate, so they silently degrade while the job still exits 0, giving a falsely reassuring pass.
This also contradicts the repo's own guidance in README.md:41 (private callers need contents: read, actions: read, pull-requests: read).
Suggested:
permissions:
security-events: write
id-token: write
contents: read
actions: read
pull-requests: readand mirror it in the two example callers and the README permissions table.
| -w /workspace \ | ||
| "ghcr.io/ossf/scorecard:${{ inputs.scorecard_version }}" \ | ||
| --repo="github.com/${{ github.repository }}" \ | ||
| --checks="${{ inputs.custom_checks }}" \ |
There was a problem hiding this comment.
${{ inputs.custom_checks }} here (and results_file, file_mode, scorecard_version, github.repository elsewhere in this block, plus line 104) is expanded into the script text before bash runs, which is the script-injection pattern our CLAUDE.md calls out (never interpolate into shell commands). Callers are trusted so the blast radius is bounded, but this assembles a docker run command line, so please pass these via env: and reference "$CUSTOM_CHECKS" etc., exactly as you already do for GITHUB_AUTH_TOKEN / ENABLE_SARIF.
|
|
||
| steps: | ||
| - name: "Checkout code" | ||
| uses: actions/checkout@v6.0.2 |
There was a problem hiding this comment.
Pin drift from the rest of the repo: every other workflow is on actions/checkout@v7, this is a major version behind on @v6.0.2. actions/upload-artifact@v7.0.1 (line 146) is also pinned to a patch tag where the repo uses @v7. Please align both with the repo convention.
|
|
||
| # Upload the results to GitHub's code scanning dashboard. | ||
| # Commenting out will disable upload of results to your repo's Code Scanning dashboard. | ||
| - name: "Upload to code-scanning" |
There was a problem hiding this comment.
github/codeql-action/upload-sarif requires a SARIF file, but nothing prevents results_format: json + upload_type: dashboard, which would fail at this step. Relatedly, the artifact step above hardcodes name: SARIF file even for JSON output. Consider validating (json implies artifact-only) or documenting the constraint, and making the artifact name format-aware. Only the sarif+artifact path was exercised in the hello-world run, so this combination is currently untested.
|
|
||
| | Access | Jobs used | Level | Reason | Conditions | | ||
| | ------------------------ | ---------- | -------- | ----------------------------------------------------------------- | ---------- | | ||
| | `security-events: write` | `analysis` | Workflow | To POST new code scanning alerts based on the SARIF report | N/A | |
There was a problem hiding this comment.
This table omits contents: read (and actions: read / pull-requests: read), which private callers need per the repo's own note at README.md:41 and which the analysis job requires for checkout and for several of the checks. Please add them here and in the two example callers once the workflow permissions block is fixed.
Pull Request
Checklist
PR Type
Linked tickets
ENG-312
High level description
This request is to add a callee workflow for OpenSSF Scorecard, to generate appraisals against a repository and optionally publish the results (with a badge). This new workflow splits into two parts, the first using the upstream
ossf/scorecard-action, which isn't customisable and uses all checks and policies by default. Conversely, the second part is customisable, taking an input that defines dynamically which policies are in scope and which checks actually run. Publishing the customised results isn't feasible with the CLI alone, so I've skipped that. Results are uploaded as artefacts or to the Advanced Security dashbaord in either SARIF or JSON format.